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ABSTRACT 

Autonomous driving technology has made significant 
advances in recent years. In order to make self-driving 
cars more practical, they are required to operate safely 
and reliably even under adverse driving condition. 
The object detection based on deep learning is an 
important application in deep learning technology, 
which is characterized by its strong capability of 
features learning and feature representation compared 
with the traditional object detection method. After 
analyzing the characteristics of videos shot by the 
camera, we choose to use deep learning to train a 
vehicle detection model to detect targets in video. In 
the end, we use trained data set to control the speed 
and navigate the vehicle in crises situations. 
Conversely, not much research is going on of usage 
such networks for elaborating of real time data. The 
goal of this work is exploring, experimenting and 
providing new approaches of classification non¬ 
stationery data using neural network. 

Keywords: Autonomous, Object Detection, Deep 
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I. INTRODUCTION 

Autonomous Driving has been said to be the next big 
disruptive innovation in the years to come. 
Considered as being predominantly technology 
driven, it is supposed to have massive societal impact 
in all kinds of fields. Having a closer look at the 
history of Autonomous Driving, as explained in the 
IEEE Spectrum it can be observed that the 
technological development and main milestones of 
the autonomous driving field started already a few 
decades ago. Leading to a vast analysis of some semi- 


autonomous features, development of present 
technologies and understanding on the future 
problematic while focusing the near future in the 
connected car. Full automated car is not accepted by 
many countries and doesn’t show high accuracy in 
real life. It doesn’t mean humans are better drivers, 
human drivers face many situations where they lose 
their human ability e.g. drunk and drive, drowsiness 
etc. For improving the safety and a new method has 
been proposed. The method includes the combination 
of deep-learning object detection and human activity 
monitoring to make self -driving more reliable. 

II. RELATED WORKS 

The application of reinforcement learning on control 
and decision-making has been investigated in several 
works. Pyeatt and Howe [10] applied reinforcement 
learning to learning racing behaviours in Robot Auto 
Racing Simulator, precursor of the TORCS platform, 
both are open source racing simulators. Daniele et al. 
[11] used the tabular Q learning model to leam the 
overtaking strategies on TORCS. Riedmiller [3] 
proposed a neural reinforcement learning method, 
namely neural fitted Q-iteration (NFQ), to generate 
control strategy for the pole balancing and mountain 
car task with least interactions. In above work, both 
the state and action spaces are low dimensional. NFQ 
used the MultiLayer Perceptron (MLP) model as the 
value function for Q iteration in which the traditional 
three-layer neural network is employed under normal 
circumstances. It may fail to find a feasible solution 
with the increased status and action spaces. In recent 
years, the deep reinforcement learning has seen an 
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exciting development. One of the representative 
works is published in Nature 2015 by the researchers 
of Google DeepMind, in which the authors developed 
a novel artificial agent of a deep Q-network, based on 
convolutional neural network (CNN) and Q-leaming. 
The artificial agent learned policies directly from 
high-dimensional sensor inputs and achieved human- 
level control on the challenging domain of classic 
Atari 2600 games [8], Another exciting work by 
Google DeepMind is more convincing. AlphaGo 
defeated the world chess champion Lee Se-dol by 4:1 
[12]. The technology behind it was a combination 
reinforcement learning, and Monte Carlo tree search 
technique, supplemented with extensive training set. 



III. Proposed System 

The proposed system developed to drive cars in an 
efficient and safer way. This system does not support 
for fully automated self-drive car. Instead, we are 
likely to present the semi-automatic car for some 
CRISES situation. The semi-automatic car can be 
easily interacted with human beings in emergency. 
[(Ex) In Drunk and Driving situation, car would have 
85% of the control, the driver can still drive the car 
but within a limited speed. During turning or Heavy 
Traffic, the car will detect the traffic signal using pre¬ 
trained model and make decisions according to the 
situation]. The four phases involve mainly a sensor 
phase, an image processing phase, an object detection 
and an automated phase. 



Fig. 1 Architecture of semi-automated system 

A. Sensor phase 

Multiple sensors and cameras are attached with the car. This provides input data to the system. Data are divided 
in two types of data, Camera data and sensor data. 


i. Camera data: 

Data from camera is used for image processing. Data from the camera is high in resolution, so it is reduced to 
low resolution. Four cameras are used, three cameras are inserted outside the cars which used control the 
steering angle, and another camera is used to collect images inside the car to detect drowsiness and mobile 
usage. 

ii. Sensor data: 


Three types of sensors are used Distance sensor used to calculate distance, EEG Sensors is used to detect 
drowsiness and Alcohol detector (MQ-3) used detect the level of alcohol in the driver’s breath. 
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Fig. 2 Camera data 


Fig. 3 MQ-3 Alcohol detector 


B. Image Processing Phase 

For image processing neural networks is used. One 
advantage of using neural network is that once the 
network is trained, it only needs to load trained 
parameters afterwards, thus prediction can be very 
fast. An end-to-end learning approach is used for self¬ 
driving. The end-to-end learning takes the raw image 
as input and outputs the control signal automatically. 


The model is self-optimized based on the training data 
and there is no manually defined rules. These become 
the two major advantages of end-to-end learning: 
better performance and less manual effort. Because 
the model is self-optimized based on the data to give 
maximum overall performance, the intermediate 
parameters are self-adjusted to be optimal. 



C. Object Detection Phase 

For object detection TensorFlow object detection API is used. In order to use this API, a Images are collected 
in TFRecords for the training and testing data, two options are available. Pre-trained model is used, and then 
transfer learning to learn a new object, or learn new objects entirely from scratch. The benefit of transfer 
learning is that training can be much quicker, and the required data that you might need is much less. For this 
reason, we're going to be doing transfer learning here. Pre trained ssd_mobilenet_vl_coco model was used to 
train the Traffic signals dataset, which produced high accuracy and faster detection. 
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Fig. 5 Traffic sign detection 


D. Automated Phase 

In this phase data from sensors used for controlling 
the car i.e., each sensor is set to a threshold value, a 
driver should satisfy this in order to control the car. 
Distance sensor is used to calculate the distance 
between the object in front of the car, for an example 
minimum distance can be set to 20 meters which 
helps the car to avoid collision. EEG sensor is used to 
detect drowsiness and the alcohol detector detects the 
alcohol level of the driver if the level is high, the car 
is switched to automatic mode and speed of car is 
reduced. Since the human driver is not is his control 
so he/she lost is human ability, so the automatic car 
can perform better than the human driver. 

E. Conclusions 

Deep learning is a new breed of Machine Intelligence 
technique, which is gaining much popularity and wide 
use in various computer science fields, such as object 
recognition, speech recognition, signal processing, 
robotics, AI gaming, and so forth. The semi- 
automated designed system uses low power 
consumption and only used in crises situations. The 
proposed system is mainly useful in reducing the 
highway accidents. In future it is very useful for 
disabled person, senior citizens and reduces the 
number of road accidents. Since semi automation 
system is used, manufacturing cost will be reduced, 
and it can be used for commercial purpose. 
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